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HIERARCHICAL ASSOCIATIVE MEMORY-BASED 
CLASSIFICATION SYSTEM 

BACKGROUND OF THE INVENTION 

CROSS-REFERENCE TO RELATED APPLICATIONS 

5 This application is related to the following co-pending, commonly owned U.S. 

Patent Application: 

U.S. Patent Application Serial No. 09/295,187 entitled, METHOD AND 
APPARATUS FOR ORGANIZING, STORING AND EVALUATING ACCESS 
CONTROL LISTS, filed April 20, 1999. 

1 0 Field of the Invention 

The present invention relates generally to computer networks, and more specifi- 
cally, to a method and apparatus for configuring an associative memory device to effi- 
ciently perform matches against long input strings, such as network messages. 

Background Information 

15 A computer network typically comprises a plurality of interconnected entities that 

transmit (i.e., "source") or receive (i.e., "sink") data frames. A common type of com- 
puter network is a local area network ("LAN") which typically refers to a privately 
owned network within a single building or campus. LANs employ a data communication 
protocol (LAN standard), such as Ethernet, FDDI or Token Ring, that defines the func- 

20 tions performed by the data link and physical layers of a communications architecture 
(i.e., a protocol stack), such as the Open Systems Interconnection (OSI) Reference 
Model. In many instances, multiple LANs may be interconnected by network links to 
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form a wide area network ("WAN"), metropolitan area network ("MAN") or intranet. 
These LANs and/or WANs, moreover, may be coupled through one or more gateways to 
the well-known Internet. 

Each network entity preferably includes network communication software, which 
5 may operate in accordance with the Transmission Control Protocol/Internet Protocol 
(TCP/IP) suite of communication protocols. TCP/IP basically consists of a set of rules 
defining how entities interact with each other. In particular, TCP/IP defines a series of 
communication layers, including a transport layer and a network layer. At the transport 
layer, TCP/IP includes both the User Datagram Protocol (UDP), which is a connection- 

1 0 less transport protocol, and TCP which is a reliable, connection-oriented transport proto- 
col. When a process at one network entity wishes to communicate with another entity, it 
formulates one or more network messages and passes them to the upper layer of the 
TCP/IP communication stack. These messages are passed down through each layer of 
the stack where they are encapsulated into segments, packets and frames. Each layer also 

1 5 adds information in the form of a header to the messages. The frames are then transmit- 
ted over the network links as bits. At the destination entity, the bits are re-assembled and 
passed up the layers of the destination entity's communication stack. At each layer, the 
corresponding message headers are stripped off, thereby recovering the original network 
message which is handed to the receiving process. 

20 One or more intermediate network devices are often used to couple LANs to- 

gether and allow the corresponding entities to exchange information. For example, a 
bridge may be used to provide a "bridging" function between two or more LANs. Alter- 
natively, a switch may be utilized to provide a '"switching" function for transferring in- 
formation, such as data frames or packets, among entities of a computer network. Typi- 

25 cally, the switch is a computer having a plurality of ports that couple the switch to several 
LANs and to other switches. The switching function includes receiving network mes- 
sages at a source port and transferring them to at least one destination port for receipt by 
another entity. Switches may operate at various levels of the communication stack. For 
example, a switch may operate at layer 2, which, in the OSI Reference Model, is called 
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the data link layer and includes both the Logical Link Control (LLC) and Media Access 
Control (MAC) sub-layers. 

Other intermediate devices, commonly referred to as routers, may operate at 
higher communication layers, such as layer 3, which in TCP/IP networks corresponds to 
5 the Internet Protocol (IP) layer. IP message packets include a corresponding header 
which contains an IP source address and an IP destination address. Routers or layer 3 
switches may re-assemble or convert received data frames from one LAN standard (e.g., 
Ethernet) to another (e.g. Token Ring). Thus, layer 3 devices are often used to intercon- 
nect dissimilar subnetworks. Some layer 3 devices may also examine the transport layer 

10 headers of received messages to identify the corresponding TCP or UDP port numbers 
being utilized by the corresponding network entities. Such extended-capability devices 
are often referred to as Layer 4, Layer 5, Layer 6 or Layer 7 switches or as Network Ap- 
pliances. Many applications are assigned specific, fixed TCP and/or UDP port numbers 
in accordance with Request for Comments (RFC) 1700. For example, TCP/UDP port 

15 number 80 corresponds to the hyper text transport protocol (HTTP), while port number 
21 corresponds to file transfer protocol (ftp) service. 

Fig. 1 is a partial block diagram of a conventional Transport Layer segment 100 
corresponding to the TCP protocol in which a Network Layer packet 102 corresponding 
to the IP protocol is encapsulated. Segment 100 includes a TCP header portion 104 that 

20 includes a plurality of fields. In particular, TCP header 104 includes a source port field 
106 and a destination port field 108, among others. IP packet 102 similarly includes an 
IP header portion 1 10 that also comprises a plurality of fields. Specifically, IP header 
110 includes an IP source address (SA) field 1 12, an IP destination address (DA) field 
114, and a protocol field 116. IP packet 102, and thus segment 100, further includes a 

25 data portion 1 1 8. Headers 1 04 and 110 basically identify the local end points of the con- 
nection between the communicating entities and may also specify certain flow informa- 
tion. 
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Access Control Lists 

Some networking software, including the Internetwork Operating System (IOS®) 
from Cisco Systems, Inc. of San Jose, California, supports the creation of access control 
lists or filters. These access control lists are typically used to prevent certain traffic from 
5 entering or exiting a network. In particular, a layer 3 device may utilize an access control 
list to decide whether a received message should be forwarded or filtered (i.e., dropped) 
based on certain predefined criteria. The criteria may be IP source address, IP destination 
address, or upper-layer application based on TCP/UDP port numbers. For example, an 
access control list may allow e-mail to be forwarded, but cause all Telnet traffic to be 
10 dropped. Access control lists may be established for both inbound and outbound traffic 
and are most commonly configured at border devices (i.e., gateways or firewalls). 

To generate an access control list, a network administrator typically defines a se- 
quence of criteria statements using a conventional text editor or graphical user interface 
(GUI). As each subsequent statement is defined, it is appended to the end of the list. The 

15 completed list is then downloaded to the desired layer 3 device where it may be stored in 
the device's non- volatile RAM (NVRAM) typically as a linked list. Upon initialization, 
the device copies the access control list to its dynamic memory. When a packet is subse- 
quently received at a given interface of the device, a software module of IOS® tests the 
received packet against each criteria statement in the list. That is, the statements are 

20 checked in the order presented by the list. Once a match is found, the corresponding de- 
cision or action (e.g., permit or deny) is returned and applied to the packet. In other 
words, following the first match, no more criteria statements are checked. Accordingly, 
at the end of each access control list a "deny all traffic" statement is often added. Thus, if 
a given packet does not match any of the criteria statements, the packet will be discarded. 

25 As indicated above, access control lists are used primarily to provide security. 

Thus, for a given interface, only a single list is evaluated per direction. The lists, moreo- 
ver, are relatively short. Nevertheless, the evaluation of such lists by software modules 
can significantly degrade the intermediate device's performance (e.g., number of packets 
processed per second). This degradation in performance has been accepted mainly due to 

30 a lack of acceptable alternatives. It is proposed, however, to expand the use of access 
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control lists for additional features besides just security decisions. For example, access 
control lists may also be used to determine whether a given packet should be encrypted 
and/or whether a particular quality of service (QoS) treatment should be applied. Ac- 
cordingly, it is anticipated that multiple access control lists may be assigned to a single 
5 interface. As additional access control lists are defined and evaluated per packet, the re- 
duction in performance will likely reach unacceptable levels. 

To improve performance, some devices store access control lists in an associative 
memory, such as a ternary content addressable memory (TCAM). TCAM suppliers cur- 
rently make TCAMs up to 144 bits in width. This has proven acceptable because the to- 

10 tal number of bits being evaluated is on the order of 1 33. In particular, the message fields 
currently being evaluated by access control lists (i.e., the criteria) include IP source ad- 
dress, IP destination address, protocol, TCP/UDP source port, TCP/UDP destination port, 
virtual local area network (VLAN) identifier, differentiated services codepoint (DSCP), 
and the physical port on which the message was received. With version 4 of the Internet 

15 Protocol (IPv4), source and destination addresses are 32 bits in length. Accordingly, the 
above information, typically referred to as the flow label, adds up to approximately 133 
bits, which is less than the width of available TCAMs. 

With version 6 of the Internet Protocol (IPv6), however, network layer addresses 
are now 128 bits long. Assuming the same fields are to be evaluated, the flow labels be- 

20 ing evaluated are now approximately 336 bits long, which is more than twice the size of 
current TCAMs, It is also proposed to evaluate higher-level messages, e.g., up to layer 7, 
which is the application layer. This would further increase the amount of information, 
and thus the number of bits, being evaluated. Technical limitations, however, currently 
prevent TCAMs from being built to widths on the order of 336 bits. Even if they could 

25 be built, such large TCAMs would likely have substantial power requirements and would 
thus be highly inefficient. 

Accordingly, a need exists for a mechanism that can search long strings of data 
(e.g., 366 bits or more) at relatively high-speed. 
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SUMMARY OF THE INVENTION 

Briefly, the invention relates to a system and method for efficiently classifying 
long strings of data, such as network messages, by matching them. The system includes a 
hierarchically arranged memory structure. In the illustrative embodiment, the memory 
5 structure includes a plurality of ternary content addressable memories (TCAMs), which 
are themselves hierarchically arranged. The TCAMs are programmed in a novel manner 
with the data to be matched, such as the predefined statements of one or more access 
control lists (ACLs), so as to allow messages that are longer than the width of the 
TCAMs to nonetheless be matched. In operation, a given network message is provided 
10 (e.g., input) to the hierarchical TCAMs and the particular action to be applied to that 
message, as specified by the matching ACL statement, is returned. 

The hierarchical TCAMs include a top-level TCAM and at least one next-level 
TCAM. Each TCAM, moreover, may be associated with another memory device, such 
as a random access memory (RAM) having the same number of entries as its associated 

15 TCAM. According to the invention, the top-level TCAM is configured only to receive 
and match one or more sub-fields of the network message. In the illustrative embodi- 
ment, the longest sub-field(s) of the message (e.g., the network layer addresses) are se- 
lected for matching within the top-level TCAM. A match within the top-level TCAM 
specifies a corresponding record of its associated RAM. This record is programmed to 

20 contain an identifier that relates to, but is substantially shorter than, the sub-field that was 
searched (i.e., the network layer address). This identifier is then input into the next-level 
TCAM along with the remaining sub-fields of the message. A match within the next- 
level TCAM similarly specifies a corresponding record within its associated RAM. This 
record preferably specifies the particular action (e.g., drop, forward, encrypt, etc.) that is 

25 to be applied to the network message. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention description below refers to the accompanying drawings, of which: 
Fig. 1, previously discussed, is a block diagram of a conventional network mes- 
sage; 

5 Fig. 2 is a highly schematic block diagram of a computer network; 

Fig. 3 is a partial, functional block diagram of an intermediate network device in 
accordance with the present invention; 

Fig. 4 is a highly schematic block diagram of the hierarchical, associative memory 
structure of the device of Fig. 3; 
10 Fig. 5 is an illustration of an exemplary access control list; and 

Fig. 6 is a flow diagram of the preferred method of the present invention. 

DETAILED DESCRIPTION OF AN ILLUSTRATIVE 

EMBODIMENT 

Fig. 2 is a highly schematic illustration of a computer network 200. The network 
1 5 200 includes a plurality of servers 202-2 1 0 and a plurality of intermediate network de- 
vices 212-222, such as backbone routers, high-speed switches, etc. Each server 202-210 
is preferably coupled to two or more of the devices 212-222 by respective trunks or links 
224a-j. Server 202, for example, is coupled to device 212 by link 224a, and to device 
214 by link 224b. The devices 212-222 are similarly interconnected with each other by 
20 another set of trunks or links 226a-i. Significantly, the network 200 is also coupled to the 
well-known Internet, which may be represented by an Internet cloud 228. In particular, 
device 220 is coupled to the Internet 228 by link 230, and device 222 is coupled to the 
Internet 228 by link 232. Accordingly, network 200 allows users (not shown) who may 
be coupled to Internet cloud 228 through other networks or connections, to access any of 
25 the servers 202-2 1 0 and retrieve information posted on those servers 202-2 10. 
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It should be understood that the configuration of network 200 is meant for illus- 
trative purposes only, and that the present invention will operate with other, possibly far 
more complex, network designs or topologies. 

Fig. 3 is a partial block diagram of intermediate network device 222. Device 222 
5 preferably includes a plurality of interfaces 302a-302e that provide connectivity to the 
network 200 and the Internet 228. That is, interfaces 302a-302e are in communication 
with servers 202-210 and Internet 228. Each interface 302a-e, moreover, may be associ- 
ated with one or more physical ports (not shown). Device 222 further includes at least 
one forwarding entity 304, a central processing unit (CPU) 306, non- volatile random ac- 

10 cess memory (NVRAM) 308, dynamic memory 310 and an Access Control List (ACL) 
storage and searching device 312. CPU 306 is coupled to both the NVRAM 308 and also 
to dynamic memory 310. NVRAM 308 may contain one or more text-based access con- 
trol lists (ACLs) 3 1 8a-3 1 8e. Dynamic memory 310 may contain a plurality of applica- 
tions or other programs, such as an encryption function 314 and a logging function 3 16, 

1 5 that may be executed by CPU 306. 

Forwarding entity 304 may include a plurality of conventional sub-components 
configured to implement quality of service (QoS) treatments, such as a packet/frame clas- 
sifier 320, a scheduler 322, a shaper 324, a marker 326, a dropper 328, and a queue se- 
lector/mapper 330. The forwarding entity 304 is also coupled to the CPU 306 and the 
20 ACL storage and searching device 312. As described below, the forwarding entity 304 is 
basically configured to forward or switch network messages among the various interfaces 
302a-e of device 222. 

Device 222 may further include an access control list (ACL) translation engine 
332. ACL engine 332 is operatively coupled to NVRAM 308 for accessing the text- 
25 based ACLs 3 1 8a-e, dynamic memory 3 1 0 for processing the ACLs 3 1 8-e, and to ACL 
storage and searching device 312 for storing modified versions of the ACLs 318a-e 
therein, as described below. ACL translation engine 332 preferably comprises computer 
readable media containing executable software programs, such as software modules or 
libraries, pertaining to the methods described herein. 
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It should be understood that ACL translation engine 332 may be stored at dy- 
namic memory 310 and run on or otherwise be executed by CPU 306 or some other proc- 
essing element (not shown). Engine 332 may also be implemented in hardware through a 
plurality of registers and combinational logic configured to produce sequential logic cir- 
5 cuits and cooperating state machines. Those skilled in the art will recognize that other 
combinations of software and hardware implementations may be utilized. 

A suitable platform for intermediate network device 222 are the Catalyst 4000 
switches, Catalyst 8500® series of switch routers, and/or the Catalyst® 6000 family of 
multilayer switches all from Cisco Systems, Inc. of San Jose, California. 

10 Fig. 4 is a highly schematic illustration of the ACL storage and searching device 

312. ACL device 3 1 2 includes a message buffer 402, buffer control logic 403 and a hier- 
archically arranged associative memory structure 404. Device 312 may also include pre- 
parer logic 406. The associative memory structure 404 is preferably formed from a plu- 
rality of ternary content addressable memories (TCAMs) that are themselves hierarchi- 

1 5 cally arranged, such that there is a top-level TCAM 408, and least one next level TCAM 
410. Each TCAM 408, 410, moreover, may include another memory structure to which 
the respective TCAM 408, 410 is coupled. In the illustrative embodiment, top-level 
TCAM 408 is coupled to a first random access memory (RAM) 412, while next level 
TCAM 410 is coupled to a second RAM 414. Each TCAM 408, 410 has a plurality of 

20 entries and each entry is made up of a plurality of cells. The cells of the TCAMs 408, 
410, moreover, are associated with or assigned one of three possible values (e.g., "0", "1" 
or "don't care"). Each RAM 412, 414 includes a plurality of records, such that each 
TCAM entry specifies a particular record in its associated RAM. 

The preparser logic 406 is coupled to the forwarding entity 304 (Fig. 3) so as to 
25 receive messages therefrom. Preparser logic 406 is configured to extract one or more 

fields from the message which may then be temporarily stored in the message buffer 402. 
For example, preparser logic 406 may be configured to generate a desired flow label by 
extracting the contents of the destination and source ports, IP SA, IP DA and protocol 
fields, as well as the VLAN ID, DSCP and physical port on which the respective message 
30 was received. All of this information is preferably passed to and temporarily stored by 
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the message buffer 402. Under the control of buffer control logic 403, the message 
buffer 402, in turn, supplies one or more of these fields to the top-level TCAM 408 ? and 
the remaining fields to the next-level TCAM 410, as illustrated by arrows 416 and 418, 
respectively. The output of each TCAM of the hierarchical memory structure 404 is 
5 preferably provided to its respective RAM, as illustrated by arrows 422 and 424. The 
output of first RAM 412 is supplied to the next level TCAM 410, as illustrated by arrow 
420, while the output of second RAM 414 is returned to the forwarding entity 304. 

As shown, the memory structure 404 is hierarchically arranged in that the output 
from at least one TCAM (e.g., the top-level TCAM 408) is provided as an input to an- 

1 0 other TCAM (e.g., the next level TCAM 410). In the illustrative embodiment, the top- 
level TCAM 408 is used to search one or more of the longer fields of the subject message 
(such as the IP DA and IP SA fields). When a match is located within the top-level 
TCAM 408, it specifies a corresponding entry in the first RAM 412 that contains an 
identifier that is related to the respective field. The identifier from the first RAM 412 is 

15 then input to the next level TCAM 410 along with the remaining fields of the subject 

message. Significantly, the identifier from the first RAM 412 is substantially shorter (has 
far fewer bits) than the field that was supplied to the top-level TCAM 408. 

The top and next level TCAMs 408, 410 preferably have 512k or more rows and a 
length of 144 bits. A suitable TCAM for use with the present invention is described in 
20 co-pending U.S. Patent Appl. Ser. No. 09/130,890, filed August 7, 1998, which is hereby 
incorporated by reference in its entirety. Other TCAMs that can be used with the present 
invention are commercially available from NetLogic Microsystems, Inc. of Mountain 
View, California or from Music Semiconductors of Hackettstown, New Jersey. The 
TCAMs and their associated RAMs may be either static or dynamic. 

25 Although the preferred embodiment of the memory structure 404 is described as a 

plurality of hierarchically arranged TCAMs, those skilled in the art to which the inven- 
tion pertains will recognize that any associative memory devices, such as binary content 
addressable memories (CAMs), hash tables, etc., may be employed to achieve the ad- 
vantages of the present invention. Binary CAMs, for example, only support exact 

30 matching. They do not allow for "don't care" values. 

10 
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First, a network administrator creates one or more access control lists in a con- 
ventional manner. For example, the administrator preferably utilizes a conventional text 
editor at a management station (not shown) to create the access control lists. Fig. 5 is a 
5 highly schematic representation of text-based ACL 3 18a. Each access control list, such 
as ACL 3 18a, is given a name, such as ACL 101, and is preferably arranged in a table 
array having multiple rows and columns. Each row of the ACL, such as ACL 3 18a, cor- 
responds to an Access Control Entry (ACE) statement, such as ACE statements 502-514, 
which specify the various criteria for the ACL 3 18a. The columns of the ACL represent 

10 the specific criteria with which network messages are compared. For example, ACL 
318a includes a separate column for IP source address 516, IP destination address 518, 
TCP/UDP source port 520, TCP/UDP destination port 522 and transport protocol 524. 
Those skilled in the art will understand that additional message criteria (such as VLAN 
ID, DSCP, physical port, etc.) may advantageously be employed. ACL 318a further in- 

15 eludes an action column 526 that corresponds to the particular action that is to be applied 
to network messages matching a corresponding ACE statement. Exemplary actions in- 
clude permit, deny, permit and log, and deny and log, although other actions may be 
specified. For example, a possible action may be to execute a particular program stored 
in the non- volatile or dynamic memories of the respective device. 

20 The text-based ACLs that are to be utilized at a given intermediate device are then 

downloaded to that device in a conventional manner and stored, preferably in non- 
volatile memory. In particular, the ACLs may be maintained in memory as ASCII text or 
in other formats. For example, ACLs 3 1 8a-3 1 8e may be downloaded to device 222 by 
the network administrator and stored at NVRAM 308. Next, the network administrator 

25 preferably assigns one or more ACLs 3 1 8a-e to each interface 302a-e per direction (e.g., 
inbound or outbound). 

For example, the network administrator may assign ACL 318a (ACL 101) to in- 
terface 302a for purposes of input security control. Accordingly, upon receipt of a net- 
work message at interface 302a, it is compared with ACE statements 502-514 of ACL 
30 318a. As described below, the matching is preferably performed as a series of sequential 
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steps starting with the first ACE 502 and moving, one ACE at a time, toward the last 
ACE 5 14 in the ACL. Once a match is located, the corresponding action is returned and 
the processing stops. That is, no additional ACEs are examined. If a match is made with 
an ACE statement having a "permit" action (e.g., ACE 502), the packet is forwarded. If a 
5 match is made with an ACE statement having a "deny" action (e.g., ACE 506), the packet 
is dropped. If the matching action is "permit and log", then the respective message is 
forwarded and an entry is made in a conventional message log. Similarly, if the matching 
action is "deny and log", then the respective message is dropped and a log entry made. If 
no ACE of the subject ACL matches the message, an implicit action located at the end of 
10 the ACL, e.g., ACE 514, is typically returned (e.g., permit or deny). 

The value "x" of ACL 318a corresponds to a don't care condition. That is, the 
specified action is independent of the value at each "x" position of the message being 
tested. To define don't cares, an address (or other criteria) is typically supplied along 
with a "mask". The mask specifies which bits are significant and which are don't cares. 

15 Programming the Hierarchical Associative Memory Structure 

Fig. 6 is a flow diagram of the steps used in programming the hierarchical mem- 
ory 404 with the ACEs of a selected ACL, such as ACL 3 18a. As described above, with 
IPv6, the IP source address and IP destination address fields are each 128 bits long, the 
TCP/UDP source and destination port fields are each 16 bits long and the transport proto- 
20 col field is 8 bits long. The total number of bits, namely 296, far exceeds the width of 
most commercially available TCAMs. Furthermore, if additional fields are to be evalu- 
ated, such as VLAN ID, DSCP and physical port, the problem only becomes worse. As 
described herein, the present invention provides a solution to searching long strings, such 
as the flow labels of IPv6 TCP/IP messages. 

25 As indicated above, the IP source and destination address fields are the longest 

fields of the flow label. In the illustrative embodiment, it is these fields that are selected 
for programming into the top-level TCAM 408, while the remaining fields are pro- 
grammed into the next level TCAM 410. Since the TCAMs 408, 410 only return the first 
matching entry for a given input, the order in which the addresses and ACEs are pro- 
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grammed into the TCAMs 408, 410 is important. As a general proposition, more specific 
ACEs should be placed ahead of less specific ACEs. In other words, ACEs with large 
numbers of don't cares should generally be placed into lower entries of the TCAMs 408, 
410, so that more specific entries may be matched first. 

5 As shown at block 602, one of the first steps in programming the top-level TCAM 

408 is to examine all of the IP source addresses of the ACL and to identify "coordinate 
subfields". A coordinate subfield is basically a range of bit positions such that, for all of 
the IP source addresses, and/or destination addresses as the case may be, within the ACL, 
each IP address has either a specific value for the entire bit range or has all don't care 
1 0 values for the entire bit range. 

ACL 318a (Fig. 5), for example, includes the following six IPv6 source addresses 
in hexadecimal format: 

(1) 1362:23 11:0000:0000:0000:4612:XXXX:XXXX 

(2) 2992:4612:0000:0000:XXXX:XXXX:XXXX:XXXX 
15 (3) XXXX:XXXX:XXXX:XXXX:2201 :8909:3A22:FACA 

(4) 2992:8909:3A22:XXXX:XXXX:XXXX:XXXX:XXXX 

(5) 8526:6951:3698:0000:0000:7412:68DA:5000 

(6) 2113:9182:0000:0000:XXXX:XXXX:XXXX:XXXX 

The six IPv6 source addresses of ACL 318a have four sub-fields. Specifically, 
20 assuming the addresses have a bit range of 127-0 from left to right, a first sub-field corre- 
sponds to the bit range 127-80, since all six source addresses have either a specific value 
across this entire range or all don't cares across the entire range. A second sub-field cor- 
responds to the bit range of 79-64. A third sub-field corresponds to the bit range 63-32. 
A fourth sub-field corresponds to the bit range 31-0. For each of the addresses, the val- 
25 ues within each sub-field are either all specific values or all don't care values (i.e., "Xs"). 

As indicated at block 604, the next step is to determine the number of distinct val- 
ues, K, that each coordinate sub-field may have. The first sub-field, for example, has the 
following five distinct values: 

13 
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1362:2311:0000 

2992:4612:0000 

2992:8909:3A22 

8526:6951:3698 

5 2113:9182:0000 

plus an "other" value (i.e., XXXX:XXXX:XXXX). The second sub-field has one 
distinct value and an other value. The third sub-field has four distinct values and an other 
value. The fourth sub-field has two distinct values and an other value. After determining 
the number of distinct values, K, the next step is to compute the minimum number of bits 
10 needed to represent each distinct value, K, for each coordinate subfield, as indicated at 
block 606. This may be accomplished by using the following algorithm: 

log 2 (K+l) 

assuming an "other" value is also possible. Applying this algorithm, we find that 
the minimum number of bits needed to represent the five distinct and other values of the 
15 first sub-field is three. The number of bits needed to represent the second sub-field is 
one. The number of bits needed for both the third and the fourth sub-fields is two. 

Next, for each sub-field, a "unique coordinate value" (UCV) is assigned to each 
distinct and other value K within that sub-field, as indicated at block 608. Considering 
the first sub-field, for example, there are five distinct values as well as an other value, and 

20 these values are to be represented with three bits. The distinct value "1362:23 1 1 :0000" 
may be assigned to UCV "000". The distinct value "2992:4612:0000" may be assigned 
to UCV "001". The distinct value "2992:8909:3A22" may be assigned to UCV "010". 
The distinct value "8526:6951 :3698" may be assigned to UCV "01 1". The distinct value 
"21 13:9182:0000" may be assigned to UCV "100". The other value 

25 "XXXX:XXXX:XXXX" may be assigned to UCV "101". 

The distinct and other values of the remaining sub-fields may be assigned to the 
following unique coordinate values: 
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Sub-field 


Distinct Value 


Unique Coordinate Value 


2 


0 


0 


2 


other 


1 


3 


0:4612 


00 


3 


2201:8909 


01 


3 


0:7412 


10 


3 


other 


11 


4 


3A22:FACA 


00 


4 


68DA:5000 


01 


4 


other 


10 



As shown, the "other" values are preferably assigned the largest or highest UCV 
within the given range. 

The UCVs are then concatenated to form a plurality of "unique coordinate value 
5 sequences" (UCVSs), as indicated at step 610. Continuing with the above example, the 
first, second, third and fourth sub-fields are represented by 5, 2, 4 and 3 UCVs, respec- 
tively. Accordingly, there are 5*2*4*3 or 120 possible UCVSs that can be formed by 
concatenating these UCVs. Each UCVS will correspond to an IPv6 address, and many of 
the UCVSs will correspond to an actual IP source address specified in ACL 318a (Fig. 5). 
10 For example, UCVS "00000010" corresponds to the first IPv6 source address in ACL 
3 1 8a, 1 362:23 1 1 :0000:0000:0000 :46 1 2:XXXX:XXXX. UCVS "00 101110" corresponds 
to the second address, 2992:4612:0000:0000:XXXX:XXXX:XXXX:XXXX, UCVS 
"101 10100" corresponds to the third address, 

XXXX:XXXX:XXXX:XXXX:2201:8909:3A22:FACA, and so on. The UCVSs are then 
15 ordered from smallest (e.g., 00000000) to largest (e.g., 101 1 1 1 10), as indicated at step 
61 1. Since each "other" value was assigned the largest UCV in the given range, the or- 
dering of UCVSs from smallest to largest will place the more specific UCVSs first. In 
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particular, UCVS "00000000" corresponds to very specific IPv6 source address, namely 
1362:231 1 :0000:0000:0000:4612:3A22:FACA, while UCVS "101 10100" corresponds to 
a less specific address, namely XXXX:XXXX:XXXX:XXXX:2201:8909:3A22:FACA, 
and UCVS "10111110" corresponds to address 
5 XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX. 

As indicated at block 612, steps 602-61 1 are then preferably repeated for the IP 
destination addresses of ACL 3 1 8a. 

The top-level TCAM 408 is then loaded with IP addresses, while the first RAM 
412 is loaded with the corresponding UCVSs, as indicated at block 614. The top-level 
10 TCAM 408 is preferably loaded with IP addresses such that the corresponding UCVSs go 
from smallest to largest (i.e., in increasing order). For example, the first row of the top- 
level TCAM 408 is loaded with IP source address 

1362:231 1 .0000:0000:0000:4612:3 A22:FACA, which corresponds to the smallest UCVS 
(i.e., 00000000), and this UCVS, moreover, is loaded in the particular record at the first 
1 5 RAM 412 that is associated with the first row of TCAM 408. Thus, a match to this row 
of the top-level TCAM 408 will cause the first RAM to output the value 00000000. This 
process is repeated until all of the UCVSs and the corresponding addresses have been 
loaded into the first RAM 412 and the top-level TCAM 408. 

It should be understood that when an IP address having a sub-field of XXXX (i.e., 
20 don't cares), then the mask for the cells of that particular sub-field is set to indicate that 
the cells are don't cares. 

Each row of TCAM 408 preferably includes an additional bit or cell that is used 
to indicate whether the UCVS associated with that row is a source or destination address. 
For example, if this cell is asserted, then the UCVS is associated with a source address. 
25 If the cell is de-asserted, then the UCVS is associated with a destination address. Alter- 
natively, the computation of UCVs and UCVSs may be made across the entire set of 
source and destination addresses simultaneously. In another embodiment, a first top- 
level TCAM and associated RAM could be used for IP source addresses while a second 
top-level TCAM and associated RAM is used for IP destination addresses. 
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As shown, each of the 128-bit IPv6 source and destination addresses of the ACL 
3 1 8a has been reduced to just 8 bits. 

The next level TCAM 410 is then loaded with the criteria fields (i.e., columns 
516-524) of the ACEs 502-514 of ACL 318a, but with the source and destination IPv6 
address fields from columns 516 and 518 replaced with their corresponding UCV- 
concatenated (i.e., UCVS) versions with the TCAM mask set to don't care for each UCV 
sub-field of the UCVS that corresponds to "other" values, as indicated at step 616. For 
example, when UCVS "00101 110", which corresponds to IPv6 address 
2992:4612:0000:0000:XXXX:XXXX:XXXX:XXXX, is loaded into the next level 
TCAM 410, the mask for the last four bit positions (i.e., "1 1 10") is set to don't care. The 
ACL actions from column 526 are then loaded into second RAM 414, as indicated at 
block 618. The hierarchical memory structure 404 is now fully programmed as indicated 
by end block 620, and may be utilized by device 222 to evaluate messages. 

For example, suppose that device 222 receives a network message on interface 
302a that originated from the Internet cloud 228. The message is passed to the forward- 
ing entity 304 which provides it to the ACL storage and searching device 312. At the 
ACL storage and searching device 312, the pre-parser 406 extracts the relevant fields for 
generating a flow label and temporarily stores this flow label in the message buffer 402. 
The buffer control logic 403 then directs the message buffer 402 to input the IP source 
and destination addresses from the flow label into the top-level TCAM 408 either simul- 
taneously or sequentially via arrow 416. 

If the top-level TCAM 408 detects a match to the input address, a corresponding 
record in the first RAM 412 is specified, as shown by arrow 422. As discussed above, 
this record contains the UCVS derived for the matching address. Furthermore, although 
the input address may have been 128 bits long, the corresponding UCVS is far shorter 
(i.e., only 8 bits long). The UCVS for both the IP source address and the IP destination 
address are then input to the next-level TCAM 410 via arrow 420. Buffer control logic 
403 also directs the message buffer 402 to input the remaining fields from the flow label 
(e.g., TCP/UDP source and destination ports, protocol, VLAN ID, DSCP, and physical 
port) into the next-level TCAM 410 via arrow 418. Since the IP source and destination 
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addresses have been effectively translated into their much shorter UCVSs, all of this data 
is now able to fit within the width of the next-level TCAM 410. 

If the next-level TCAM 410 detects a match to the flow label, a corresponding re- 
cord in the second RAM 414 is specified, as shown by arrow 424. As discussed above, 
5 this record contains the action for the message. This action is then passed by the second 
RAM 414 to the forwarding entity 304 which carries out the specified action (e.g., per- 
mit, drop, permit and log, drop and log, etc.) on the message. 

To improve performance, the hierarchical memory structure 404 may be utilized 
to examine two or more messages substantially simultaneously. More specifically, at the 
1 0 same time that the UCVSs and remaining fields corresponding to the flow label of a first 
message are being used to search the next level TCAM 410, the IP addresses from the 
flow label of a second message may be input into the top-level TCAM 408 in order to 
identify their corresponding UCVSs. In this embodiment, there may be multiple message 
buffers, buffer controls and pre-parser logic circuits. 

1 5 It should be understood that the length of the UCVSs for an ACL having more 

entries than ACL 318a may be significantly greater than 8 bits. In a preferred embodi- 
ment, UCV sequences on the order of 32 bits in length are derived and utilized within 
ACL storage and searching device 312 in the manner described above. 

It should be further understood that the translation of IP addresses to their UCVSs 
20 and the subsequent programming of the ACL storage and searching device 3 12, as de- 
scribed above with reference to Fig. 6, may be performed remotely from device 222. For 
example, these functions may be performed at a management station and the results re- 
motely programmed into the ACL storage and searching device 3 12. In this case, there 
would be no need for the device 222 to include an ACL translation engine 332. 

25 Depending on the length of the data string being searched, moreover, the hierar- 

chical associative memory 404 may contain additional TCAM levels. In other words, the 
output of next-level TCAM 410 may be input to yet another TCAM along with still fur- 
ther fields from the message buffer, and so on. At each level (except for the last and final 
level), one or more fields of the data string are converted to their corresponding UCVSs 
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for inputting into the next lower TC AM, As a result, the hierarchical associative memory 
404 of the present invention may be able to search data strings that are far longer than the 
individual TCAMs are wide. 

It should also be understood that the output of the top-level (or any other) TCAM 
5 may be processed (e.g., subjected to some algebraic expression or function), thereby gen- 
erating some new value that is input to the next-tevel TCAM. In other words, the output 
of one TCAM may be directly or, as described here, indirectly coupled to the next-level 
TCAM. Similarly, one or more fields of the data string may be pre-processed generating 
a derived value which is then input to the top-level (or any other) TCAM, Thus, the in- 
10 puts to a TCAM level may consist of any combination of: the output of a higher level 
TCAM, the processed output of a higher level TCAM, one or more fields of the data 
string, and one or more values derived from one or more fields of the data string. 

The foregoing description has been directed to specific embodiments of this in- 
vention. It will be apparent, however, that other variations and modifications may be 
1 5 made to the described embodiments, with the attainment of some or all of their advan- 
tages. For example, the techniques of the present invention may be applied to searching 
other long data strings such as URLs or other data records or files within an associative 
memory structure. Therefore it is an object of the appended claims to cover all such 
variations and modifications as come within the true spirit and scope of the invention. 

20 What is claimed is : 
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CLAIMS 

1 1 . A hierarchical memory structure for use in matching a data string having a plu- 

2 rality of fields, the associative memory structure comprising; 

3 a top-level associative memory; and 

4 at least one nexMevcl associative memory operably coupled to the top-level asso- 

5 ciative memory so as to receive an output therefrom, wherein 

6 the top-level associative memory is configured to receive and match one 

7 or more of the fields of the data string and, in response to detecting a match, pro- 

8 viding an output to the next-level associative memory, and 

9 the at least one next-level associative memory is configured to receive and 

10 match the output from the top-level associative memory plus one or more other 

1 1 fields of the data string or one or more values derived therefrom and, in response 

1 2 to detecting a match, outputting a result. 

1 2. The hierarchical memory structure of claim 1 wherein the data strings being 

2 matched are network messages, 

1 3, The hierarchical memory structure of claim 2 wherein the one or more fields 

2 input to the top-level associative memory include an Internet Protocol (IP) address field. 

1 4. The hierarchical memory structure of claim 1 wherein the top-level associative 

2 memory and next-level associative memory are each ternary content addressable memo- 

3 ries (TCAMs) that support don't care values, 

1 5. The hierarchical memory structure of claim 4 wherein the data strings being 

2 matched arc network messages. 
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1 6. The hierarchical memory structure of claim 5 wherein the one or more fields 

2 input to the top-level associative memory include an Internet Protocol (IP) address field. 

1 7. The hierarchical memory structure of claim 1 wherein the output of the top- 



2 level associative memory that is received by the at least one next-level associative mem- 

3 ory has a length that is shorter than a length of the one or more fields of the data string 

4 that are input into the top-level associative memory. 



1 8. The hierarchical memory structure of claim 7 wherein the length of the output 

2 of the top-level associative memory is on the order of one-fourth or less of the length of 

3 the one or more fields of the data string that are input into the top-level associative mem- 

4 ory. 

1 9. The hierarchical memory structure of claim 8 further comprising a first mem- 

2 ory structure associated with the top-level associative memory, the top-level associative 

3 memory and first memory structure configured such that, in response to detecting a match 

4 to an entry in the top-level associative memory, the top-level associative memory speci- 

5 fies a location of the first memory structure containing the output that is provided to the 

6 at least one next level associative memory. 



1 10. The hierarchical memory structure of claim 9 further comprising a message 

2 buffer for temporarily storing the data string, and buffer control logic for providing se- 

3 lected fields from the message buffer to the top-level and next-level associative memo- 

4 ries. 

1 11. The hierarchical memory structure of claim 1 further comprising: 
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2 a first memory structure associated with the top-level associative memory, the 

3 top-level associative memory and first memory structure configured such that, in re- 

4 sponse to detecting a match to an entry in the top-level associative memory, the top-level 

5 associative memory specifies a location of the first memory structure containing the out- 

6 put that is provided to the at least one next level associative memory; and 



7 a second memory structure associated with the at least one next-level associative 

8 memory, the next-level associative memory and second memory structure configured 

9 such that, in response to detecting a match to an entry in the next-level associative mem- 

10 ory, the next-level associative memory specifies a location of the second memory struc- 

1 1 ture containing information related to the matching next-level associative memory entry. 

1 12. The hierarchical memory structure of claim 1 1 wherein the first and second 

2 memory structures are random access memories (RAMs). 

1 13. The hierarchical memory structure of claim 12 wherein the data strings being 

2 matched are network messages and the one or more fields input to the top-level associa- 

3 five memory include an Internet Protocol (IP) address field. 

1 14. The hierarchical memory structure of claim 13 wherein the output of the first 

2 RAM has a length that is shorter than the IP address input into the top-level associative 

3 memory. 

1 15. The hierarchical memory structure of claim 14 wherein the length of the out- 

2 put of the top-level associative memory is on the order of one-fourth or less of the length 

3 of the one or more fields of the data string that are input into the top-level associative 

4 memory. 
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1 1 6. The hierarchical memory structure of claim 4 further comprising a plurality of 

2 records organized into common fields, wherein 

3 at least one field of each record, which originally contained a corresponding 

4 value, is replaced with a unique coordinate value (UCV) that has been generated for the 

5 corresponding value, and 

6 each UCV is shorter than the corresponding value that it replaces. 



1 17. The hierarchical memory structure of claim 1 further comprising a plurality of 

2 records organized into common fields, wherein 

3 at least one field of each record, which originally contained a corresponding 

4 value, is replaced with a unique coordinate value (UCV) that has been generated for the 

5 corresponding value, and 

6 each UCV is shorter than the corresponding value that it replaces. 



1 1 8 . A method for loading a hierarchical, associative memory structure with a plu- 

2 rality of records, each record organized into common fields having values and/or don't 

3 cares, so that a data string, also having a plurality of fields, may be compared with the 

4 contents of the memory structure in order to identify a matching record, the method com- 

5 prising the steps of: 

6 identifying the coordinate sub-fields of at least one selected field of the records, 

7 the selected field having distinct values or don't cares; 

8 deterrriining the number of distinct values that each coordinate sub-field has; 

9 for each coordinate sub-field, computing the minimum number of bits needed to 

1 0 individually represent each of the distinct values and don't care, if present, for the re- 

1 1 spective coordinate sub-field; 

12 assigning a unique coordinate value (UCV), that falls within the previously com- 

1 3 puted minimum number of bits, for each distinct value and don't care, if present; 
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14 for each record, generating a unique coordinate value sequence (UCVS) by con- 

1 5 catenating the UCVs assigned to the distinct values and don't care, if present, of the re- 

1 6 spective record; and 

17 loading the hierarchical, associative memory structure with the generated UCVSs. 

1 19. The method of claim 18 wherein a coordinate sub-field is defined as having 

2 either all distinct values or all don't cares for all records. 

1 20. The method of claim 1 9 wherein the records are access control entries (ACEs) 

2 of at least one access control list (ACL). 

1 21 . The method of claim 20 wherein the selected field corresponds to an Internet 

2 Protocol (IP) address field. 



1 22. An apparatus for loading a hierarchical, associative memory structure with a 

2 plurality of records, each record organized into common fields having values and/or don't 

3 cares, so that a data string, also having a plurality of fields, may be compared with the 

4 contents of the memory structure in order to identify a matching record, the method com- 

5 prising the steps of: 

6 means for identifying the coordinate sub-fields of at least one selected field of the 

7 records, the selected field having distinct values or don't cares; 

8 means for determining the number of distinct values that each coordinate sub- 

9 field has; 

10 for each coordinate sub-field, means for computing the minimum number of bits 

1 1 needed to individually represent each of the distinct values and don't care, if present, for 

12 the respective coordinate sub-field; 
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13 means for assigning a unique coordinate value (UCV), that falls within the previ- 

14 ously computed minimum number of bits, for each distinct value and don't care, if pres- 

15 ent; 

16 for each record, means for generating a unique coordinate value sequence 

17 (UCVS) by concatenating the UCVs assigned to the distinct values and don't care, if pre- 

1 8 sent, of the respective record; and 

19 means for loading the hierarchical, associative memory structure with the gener- 

20 atedUCVSs. 

1 23. A computer readable medium containing executable program instructions for 

2 loading a hierarchical, associative memory structure with a plurality of records, each rec- 

3 ord organized into common fields having values and/or don't cares, so that a data string, 

4 also having a plurality of fields, may be compared with the contents of the memory 

5 structure in order to identify a matching record, the executable program instructions 

6 comprising steps for: 

7 identifying the coordinate sub-fields of at least one selected field of the records, 

8 the selected field having distinct values or don't cares; 

9 determining the number of distinct values that each coordinate sub-field has; 

10 for each coordinate sub-field, computing the minimum number of bits needed to 

1 1 individually represent each of the distinct values and don't care, if present, for the re- 

12 spective coordinate sub-field; 

13 assigning a unique coordinate value (UCV), that falls within the previously com- 

1 4 puted minimum number of bits, for each distinct value and don't care, if present; 

15 for each record, generating a unique coordinate value sequence (UCVS) by con- 

16 catenating the UCVs assigned to the distinct values and don't care, if present, of the re- 

1 7 spective record; and 

1 8 loading the hierarchical, associative memory structure with the generated UCVSs. 
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ABSTRACT OF THE DISCLOSURE 

A system and method for efficiently searching long strings of data, such as net- 
work messages, is described. The system preferably includes an associative memory 
structure, having a plurality of content addressable memories (CAMs). The CAMs are 

5 hierarchically arranged such the output of at least one CAM is used as the input to a sec- 
ond CAM. Preferably, a top-level CAM receives only a selected portion of the data 
string or network message as its input. The output of the top-level CAM is then joined 
with some or all of the remaining portions of the data string to form a new output that is 
provided to the CAM at the next lower level. The top-level CAM is programmed such 

10 that its output is substantially smaller (e.g., has fewer bits) than the selected data string 
portion that is input to the top-level CAM. The system can thus search data strings that 
are on the whole far longer than the widths of the respective CAMs forming the memory 
structure. 
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EXAMINE THE IP SOURCE ADDRESSES OF THE SUBJECT ACL AND IDENTIFY THEIR 

COORDINATE SUB-FIELDS 






DETERMINE THE NUMBER OF DISTINCT VALUES, K, THAT EACH COORDINATE SUB- 
FIELD MAY HAVE 






FOR EACH COORDINATE SUB-FIELD, COMPUTE THE MINIMUM NUMBER OF BITS 
NEEDED TO REPRESENT EACH DISTINCT VALUE, K 






ASSIGN A UNIQUE COORDINATE VALUE TO EACH DISTINCT VALUE, K, 






CONCATENATE THE UNIQUE COORDINATE VALUES TO FORM A PLURALITY OF UNIQUE 

COORDINATE VALUE SEQUENCES 






ORDER THE UNIQUE COORDINATE VALUE SEQUENCES FROM SMALLEST TO LARGEST 






REPEAT STEPS 602 THROUGH 611 FOR THE IP DESTINATION ADDRESSES OF THE 

SUBJECT ACL 






LOAD THE FIRST RAM WITH THE UNIQUE COORDINATE VALUE SEQUENCES AND LOAD 
THE TOP-LEVEL TCAM WITH THE CORRESPONDING ADDRESSES 






LOAD THE NEXT-LEVEL TCAM WITH THE ACEs OF THE SUBJECT ACL, REPLACING THE 
ORIGINAL IP SOURCE AND DESTINATION ADDRESSES WITH THE PREVIOUSLY 
GENERATED UNIQUE COORDINATE VALUE SEQUENCES FOR THESE ADDRESSES 






LOAD THE ACL's ACTIONS INTO THE SECOND RAM 
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